Modeling and Evaluating Pilot Performance in NextGen: Review of and Recommendations Regarding Pilot Modeling Efforts, Architectures, and Validation Studies

NextGen operations are associated with a variety of changes to the national airspace system (NAS) including changes to the allocation of roles and responsibilities among operators and automation, the use of new technologies and automation, additional information presented on the flight deck, and the entire concept of operations (ConOps). In the transition to NextGen airspace, aviation and air operations designers need to consider the implications of design or system changes on human performance and the potential for error. To ensure continued safety of the NAS, it will be necessary for researchers to evaluate design concepts and potential NextGen scenarios well before implementation. One approach for such evaluations is through human performance modeling. Human performance models (HPMs) provide effective tools for predicting and evaluating operator performance in systems. HPMs offer significant advantages over empirical, human-in-the-loop testing in that (1) they allow detailed analyses of systems that have not yet been built, (2) they offer great flexibility for extensive data collection, (3) they do not require experimental participants, and thus can offer cost and time savings. HPMs differ in their ability to predict performance and safety with NextGen procedures, equipment and ConOps. Models also vary in terms of how they approach human performance (e.g., some focus on cognitive processing, others focus on discrete tasks performed by a human, while others consider perceptual processes), and in terms of their associated validation efforts. The objectives of this research effort were to support the Federal Aviation Administration (FAA) in identifying HPMs that are appropriate for predicting pilot performance in NextGen operations, to provide guidance on how to evaluate the quality of different models, and to identify gaps in pilot performance modeling research, that could guide future research opportunities. This research effort is intended to help the FAA evaluate pilot modeling efforts and select the appropriate tools for future modeling efforts to predict pilot performance in NextGen operations

The Impact of Human Systems Integration on Major Defense Acquisition Program Success

This investigative study demonstrated the benefits of addressing human considerations during the system development life cycle in order to have had long-term benefits to program managers and systems engineers. The approach was to use a retrospective content analysis of documents from weapon system acquisition programs, namely Major Defense Acquisition Programs, in order to seek the presence of terminology relating to Human Systems Integration. There is only a small amount of published research to date on the relationship between program documents that included terminology relating to Human Systems Integration and any eventual cost change or schedule change for Department of Defense weapon systems. Binary logistic regression analyses were conducted to investigate the effect of the presence of words relating to Human Systems Integration on the success of programs. The presence of terminology about human factors engineering, habitability, and survivability in a weapon system acquisition program’s documents was a good indicator that schedule slippages and cost overruns would be avoided. Furthermore, the presence of terminology about human factors engineering, habitability, and survivability in a program’s documents prior to the Milestone B decision point was a good indicator that schedule slippages and cost overruns would be avoided